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ABSTRACT 

Although  tne  strttcgic  airlift  system  is  under  continuous  analysis,  C-5A  problr  ns 
provided  impetus  to  analyte  the  airlift  system  productivity  function  by  using  a  large-s.ale 
simulation  model  ./Development  of  the  simulation  model  (Simulation  of  Airlift  Resources- 
SOAR)  was  initialed  by  the  Office  of  Secretary  of  Defense  (Systems  Analysis)  in  1966.  SI  AR 
had  barely  become  operational  in  time  for  the  study  in  November  1968. 

Since  limited  verification  and  validation  tests  had  been  performed  on  the  simulation 
model,  the  design  of  experiments  was  of  critical  importance.  The  experimental  design  had 
to  br  flexible  enough  to  salvage  'he  maximum  amount  of  information  possible  upon  the 
discovery  of  either  a  verification  or  validation  error.  In  addition,  the  experimental  design 
was  required  to  accommodate  the  estimation  of  a  large  number  of  possibly  changing 
independent  variables, 

The  >  jtperimenti  1  design  developed  for  the  analysis  was  full  factorial  design  sets  for  a 
finite  number  of  factors.  Initial  analysis  began  with  »cj£regated  sets  of  factors  at  two  levels, 
and  information  galnel  from  experiment  execution  was  used  to  parse  the  sets.  The  process 
was  sequential  and  parsing  continued  until  the  major  explanatory  independent  variables  were 
identified  tit  enough  if.  formation  was  obtained  to  eliminate  the  factor  from  further  direct 
analysis.  This  design  permitted  the  overlapping  of  simulation  runs  to  fill  out  the  factorial 
design  sets.  •  ' 

In  addition  to  estimating  the  airlift  productivity  function  several  other  findings  are 
reported  which  tended  to  disprove  previous  assumptions  about  the  nature  of  the  strategic 
airlift  system. 

PRODUCTIVITY  ESTIMATES  OF  THE  STRATEGIC  AIRLIFT  SYSTEM  USING 
SIMULATION 

Computer  simulation  was  used  in  a  recent  Department  of  Defense  stucy  designed  to  evaluate 
the  impact  of  strategic  airlift  system  characteristics  on  the  productivity  of  the  military  ai  ’  t  system.* 
Although  the  system  is  under  continuous  analysis,  budgetary  pressures  and  adverse  :•  action  to  the 
C~5A  cost  increases  prompted  the  formation  of  a  joint  OSD/ Air  Force  study  group  to  evaluate  a  proposal 

♦Toe  Mobility  Forces  Division  of  Systems  Analyst  in  the  Office  of  the  Secretary  of  Defense  perfumed  the  analysis  with 
Headquarter*.  United  Stats*  Air  Force. 

The  simulation  analysis  of  the  strategic  siiUft  syt  em  ia  one  component  of  an  optimlxstion/simulstion  framework  for  the 
analysis  of  strategic  mobility.  For  a  description  of  the  fiamework  see  R.  L.  Nolan  and  M.  G.  Sovereign.  “A  Recursive  Optimisa¬ 
tion  and  Simulation  Approach  to  Analysis  with  an  App  ication  to  Transportation  Systems,"  Management  Science  (August  1972). 
For  a  description  of  the  simulation  analysis  of  the  str»t<  gic  sealift  component,  tee  R.  I..  Nolan  and  M.  G.  Sovereign.  "Simulation 
of  the  Strategic  Sealift  System,”  Proceeding}  of  IV? I  Summer  Computer  Simulation  Conference,  pp.  1184-1191. 
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to  reduce  procurement  of  the  controversial  Lockheed  transport  aircraft.  The  hypothesis  had  been  made 
that  with  a  marked  increase  in  the  daily  utilization  rate  (i.e.,  flying  hours  per  day)  of  aircraft  in  the 
transport  force,  wartime  deployment  capability  could  be  maintained  at  the  required  level  with  sig¬ 
nificantly  fewer  C-SAs.  A  controversy  developed  over  the  feasibility  of  operating  the  military  aircraft 
force  at  the  proposed  higher  utilization  rate  under  conditions  of  a  wartime  rapit2  deployment.  Of  par¬ 
ticular  concern  was  the  uncertainty  over  the  potential  contribution  to  higher  utilization  rates  of  increases 
in  airlift  support  resources.  Previous  tests  and  analytical  efforts  to  determine  feasible  utilization  rates 
were  inconclusive  in  resolving  the  question  of  increased  resources  or  in  supporting  or  rejecting  the 
hypothesis. 

A  useful  analysis  of  utilization  rates,  and  its  derivative  productivity,  must  consider  not  only  the 
factors  affecting  utilization  rates,  but  factor  interactions  and  combined  impact  on  the  airlift  system. 
Because  of  the  number  of  variables  and  the  complexity  of  the  system,  simulation  emerged  as  the  most 
appropriate  method  of  analysis.  In  1966  the  Office  of  Assistant  Secretary  of  Defense  (Systems  Analysis) 
had  initiated  the  development  of  several  computer  models  to  aid  strategic  mobility  planning  and 
analysis.  One  of  these  models  was  a  Simulation  of  Airlift  Resources  (SOAR),  a  computer  simulation 
of  intertheather  airlift  operations  under  conditions  of  rapid  deployment  which  had  been  designed  and 
developed  within  the  Office  of  Joint  Chiefs  of  Staff  (SASM).t 

The  complexity  of  the  system  and  the  decision  to  use  simulation  posed  two  different  problems  in 
the  design  of  the  analysis: 

1.  Model  verification  and  validation. 

2.  Experimental  design. 

The  proper  resolution  of  both  problems  is  key  to  developing  useful  conclusions.  Complete  verification 
of  a  large  simulation  model  such  as  SOAR  is  almost  impossible.  Validation  in  the  sense  of  testing  the 
“goodness  of  fit"  of  the  model  with  real  world  data  was  infeasible  for  the  following  reasons: 

1.  SOAR  is  one  of  the  largest  computer  simulations  designed. 

2.  The  strategic  airlift  system  has  never  been  exercised  in  an  intertheater  rapid  deployment. 

3.  Data  from  previous  exercises  of  the  strategic  airlift  system  are  (a)  incomplete,  and  (b)  obsolete. 

4.  Field  test  data  on  parts  of  the  system  are  difficult  to  interpret  because  they  were  not  collected 
for  the  purposes  at  hand,  as  well  as  the  fact  that  environmental  conditions  are  rarely  controlled  in 
such  tests. 

The  task  of  verification  and  validation  was  redefined  within  the  constraints  outlined.  The  objective 
of  the  “verification  and  validation”  process  is  to  develop  a  “threshold  of  confidence,”  which  can  be 
conveyed  to  decisionmakers,  that  the  model  is  a  sufficient  abstraction  of  reality  to  be  used  as  a  basis 
for  decisions.  A  number  of  techniques  are  available  for  building  this  level  of  confidence.  One  of  the 
most  important  was  the  involvement  of  operations  managers  in  the  design  of  the  mode!  and  interpre¬ 
tation  of  model-generated  results.  In  addition,  statistical  analysis  of  the  model  results  provided  a 
measure  of  consistency  and  inherent  variability  in  the  model  itself,* 


t  Office  of  the  Joint  Chief*  of  Staff  (Special  Assistant  for  Strategic  Mobility). 

*  Naylor  and  Finger  together  with  a  critique  by  McKenney  atate  the  validation  problem  end  sum.nerue  the  approaches 
to  it.  See  Thoma*  H.  Naylor  and  J.  M.  Finger,  “Verification  of  Computer  Simulation  Model*,”  Managtmtnt  Science  (October 
t%7),  and  James  L.  McKenney,  “Critique  of  Verification  of  Computer  Simulation  Models"  in  the  same  issue. 
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SOAK  Model 

SOAR  is  programmed  by  usings  a  derivative  of  the  GASP  simulation  language,  which  is,  it?  turn, 
a  derivative  of  FORTRAN.  It  can  simulate  up  to  1 .000  aircraft,  100  airbases,  and  1.500  air  crews. 
Approximately  6  to  7  minutes  of  computer  (HIM  360/65)  time  arc  required  to  simulate  the  activities 
of  1  day. 

In  SOAR  the  important  categories  of  variables  in  the  strategic  airlift  system  are  represented. 
Air  traffic  density  is  a  function  of  the  number  of  mission  aircraft  (C— 141  and  C-5A)  and  the  competing 
aircraft  'Civil  Reserve  Air  Fleet  (CRAF).  tactical  airlift,  fighters,  reconnaissance).  Weather  effect 
is  simulated  by  specifying  the  effect  of  seasonal  enroute  winds  on  leg  flying  time,  fuel  requirements 
and  allowable  cabin  load.  In  addition,  hourly  weather  observations  for  each  base  in  the  simulation 
can  close  a  base  when  conditions  of  ceiling,  visibility,  or  crosswind  dictate.  All  base  resources,  com¬ 
prising  a  third  category  of  resources,  are  treated  in  detail.  This  category  includes  the  personnel  and 
equipment  for  cargo  handling,  maintenance,  and  refueling,  aircrews,  parking  space,  and  airfield 
acceptance  data.  Airfield  acceptance  data  specify  base  IFR  (Instrument  Flying  Rules)  minima  and 
the  number  of  aircraft  per  hour  which  can  be  managed  by  approach  and  landing  controllers.  A  fourth 
category  of  variables  addressed  by  the  computer  model  includes  those  aircraft  characteristics  (e.g., 
speed,  fuel  consumption  rates,  parking  “shadow”)  which  impact  system  performance.  Other  system 
characteristics  which  may  have  an  effect  such  as  crew  staging  policies,  routing  policies,  scheduled 
maintenance  policies,  and  warning  time  are  addressed  both  in  problem  formulation  and  in  ordering 
of  input  data. 

Each  aircraft  to  be  simulated  is  located  at  a  specific  base  and  given  a  starting  time.  The  cargo  t 
be  moved  is  identified  by  weight  and  its  location  at  bases.  The  effect  of  cargo  density  is  described 
by  a  cumulative  frequency  distribution  which  essentially  describes  the  probability  of  a  C-5A  or  a 
0141  being  loaded  at  a  given  weight.  Levels  of  maintenance,  refueling,  cargo  handling,  and  aircrew 
capability  ure  specified  for  each  base.  With  the  scenario  and  system  defined,  initialization  is  begun. 

During  initialization,  the  input  data  are  checked  for  internal  consistency  and  completeness,  and 
the  random  number  generator  is  used  to  establish  the  initial  values  of  those  aircraft,  aircrew  and 
resource  attributes  which  are  stochastic  in  nature.  Following  these  actions,  the  simulation  is  ready 
to  begin. 

In  SOAR,  there  are  four  types  of  buses  (home,  onload,  enroute,  and  offload),  and  at  each  base  are 
located  resources  consistent  with  its  function.  For  instance,  at  onload  and  offload  bases  there  can  be 
found  aerial  port  crews  needed  to  process  cargo:  at  home  bases,  a  large  pool  of  aircrews  and  the 
facilities  necessary  for  all  maintenance  short  of  depot-level;  at  all  bases,  of  course,  are  parking  spaces, 
transient  maintenance  and  fuel.  The  simulation  begins  with  each  aircraft  at  its  home  base.  There 
the  uircruft  is  checked  to  ascertain  that  it  iias  no  maintenance  problems,  an  aircrew  is  ready,  and 
adequate  fuel  is  aboard,  if  weather  permits,  the  aircraft  departs  lor  the  airfield  that  has  the  highest 
priority  cargo  awaiting  pick-up.  On  the  aircraft's  arrival  at  the  onload  base,  the  model  selects  the 
route  to  be  taken  from  this  onload  base  to  the  cargo  destination  (offload  base).  The  route  selection  is 
based  on  factors,  such  as  weight  of  the  ourgo,  fuel  requirements,  weather,  and  the  availability  of 
resources  at  the  bases  in  the  several  candidate  route  networks.  With  the  route  selected  and  the  cargo 
now  placed  aboard,  it  cheek  is  made  to  assure  that  the  aircraft  is  still  operationally  reedy  in  terms 
of  maintenance,  aircrew,  and  fuel.  Deficiencies  are  corrected,  and  the  aircraft  departs  on  its  next 
leg  weather  permitting. 
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The  procedure  is  repeated  as  the  aircraft  moves  through  its  enroute  bases  and  arrives  at  the 
delivery  point.  Aft<v  cargo  is  offloaded,  the  model  determines  which  onload  base  has  an  appropriate 
load  of  cargo  to  be  picked  up  by  this  aircraft.  Again  necessary  maintenance  and  fuel  servicing  is 
performed,  a  rested  aircrew  replaces  the  one  aboard  if  required,  a  weather  check  is  performed  and 
the  aircraft  departs  for  its  next  onload  base  to  recommence  the  cycle. 

As  an  aircraft  moves  through  the  airlift  system,  from  home  to  onload  through  a  series  of  enroute 
bases  and  finally  to  offload,  it  makes  demands  on  the  resources  available  at  these  bases.  Where  de¬ 
mands  exceed  the  capability  of  the  bases  to  meet  them,  queues  form  adding  to  ground  time  reflecting 
the  penalties  resulting  from  system  saturation.  Some  of  the  demands  are  deterministic;  aircrews  are 
replaced  if  duty  time  will  exceed  18  hours,  fuel  requirements  are  based  on  flying  time  and  fuel  con¬ 
sumption  rates,  and  the  parking  area  used  is  the  result  of  “parking  shadow”  of  the  aircraft  itself. 

Other  demands  are  stochastic:  maintenance  failures  are  generated  randomly  from  a  distribution 
as  a  maintenance  man-hours  required  to  repair  the  failure,  the  routes  themselves  are  selected  by 
the  random  process  which  determines  the  weight  of  the  cargo  loaded  on  each  aircraft.  Still  others  are 
a  combination  in  which  a  random  number  generation  selects  an  entry  value  in  a  table  which  then  is 
processed  in  a  deterministic  manner. 

However  arrived  at,  it  is  these  demands  an1!  the  ability  of  supporting  systems  to  satisfy  them 
which  are  key  elements  of  the  airlift  simulation.  Since  only  the  lack  of  one  or  more  of  the  essential 
resources  will  hold  aircraft  on  the  ground,  SOAR  is  well  designed  for  the  evaluation  of  the  impact  of 
changing  levels  of  system  resources  and  demands  on  aircraft  utilization  rates. 

Design  of  the  Experiments 

SOAR  is  typical  of  the  large-scale  simulation  model  It  consists  of  many  instructions,  complex 
logic,  and  is  expensive  to  run.  Eventually,  the  model  grows  beyond  the  ability  of  the  analyst  to  compre¬ 
hend  all  of  the  interactions  between  the  components.  Hunter  and  Naylor*  suggest  the  use  of  a  two-stage 
experimental  design  for  analysis  on  the  large  scale  simulation  model.  The  first  stage  can  be  termed 
exploratory.  The  exploratory  stage  is  intended  to  discover  the  major  contributing  independent  vari¬ 
ables  to  the  response,  or  dependent  variable.  Techniques  such  as  factorial  analysis  are  especially 
useful  along  with  other  tools  such  as  regression  analysis,  analysis  of  variance,  and  linear  mathematical 
models.  The  exploratory  stage  is  followed  by  an  optimization  stage  which  is  intended  to  optimize  the 
response  variable,  or  at  least  improve  it,  by  adjusting  the  controllable  independent  variables. 

The  experimental  design  reported  in  this  paper  focuses  on  the  exploratory  stage.  The  response 
variable  is  aircraft  utilization  and  aircraft  productivity,  each  defined  for  the  C-5A  and  €-141.  In 
general  the  response  variables  {Y)  are  a  function  of  the  factors  (Ai),t  that  is 

Y-f(XuXt . *«). 

*  J.  S.  Hunter  and  Thomas  H.  Naylor.  "Experimental  Design,"  in  Thomas  H.  Naylor,  editor.  Thr  Design  of  Computer  St.%u- 
lotion  Experiments  (Durham,  North  Carolina;  Duke  University  Press,  1969).  pp.  39-58.  Much  has  hcetl  written  in  general  c-i 
experimental  design  techniques.  See  for  example.  D.  It.  Cox.  Claiming  of  Exi/eriments  (New  York;  John  Wiley  and  Sons,  1961). 
W.  Mendenhall.  Introduction  to  Linear  Models  and  the  Design  and  Analysis  of  Experiments  (Belmont,  California:  Wadsworth 
Publishing  Company,  1968),  and  Ken  (Tien  Peng.  The  Design  Anulysis  of  Experiments  (Reading.  Mass.;  Addison,  Wesley.  1967). 
Only  relatively  recently  have  the  design  of  e.-.|>erimeiils  for  (ompuler  -in  ulalion  experiments  been  disuissed  in  the  literature. 
The  special  problems  arc  focused  around  the  expense  of  making  large  scale  computer  simulation  experiments  and  the  model 
verification/validation  problem. 

t  Since  the  analyst  may  only  set  levels  of  controlled  variables,  the  response  is  t  harai  tetiaed  as  a  font  turn  of  the  controlled 
variables. 
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where  the  Xt  are  the  variables  listed  in  Table  1.  For  any  given  computer  run  the  variables  take  on 
specific  values  xr. 

X\  —  X|,  Xi  —  Xt,  .  .  .,  A 23  =  Xi3- 


Table  1.  SOAR  Modei  Variables 


Variable 

number 

Variable  description 

1 

Mission  aircraft 

Competing  aircraft -other  tactical  aircraft,  fighters,  etc. 

mm 

Personnel  for  cargo  handling 

Equipment  for  cargo  handling 

5 

Maintenance  personnel 

6 

Maintenance  equipment 

Refueling  personnel 

Refueling  equipment 

10 

Aircraft  parking  space  at  a  base 

11 

Airfield  capacity 

12 

Weather  data-including  the  effects  of  seasonal  enroute  winds,  fuel  requirements. 

allowable  cabin  load 

13 

Hourly  weather  data  about  the  conditions  at  a  particular  base 

14 

Aircraft  speed 

15 

Aircraft  fuel  consumption 

16 

Aircraft  parking-landing  requirements 

17 

Aircraft  maintenance  data 

,»8 

Crew  singing  policies  at  bases 

19 

Routes  and  routing  i>ol!cies 

20 

Scheduled  maintenance  policy 

21 

System  alert  warning  time 

22 

Airbase  locations  and  descriptions 

23 

Other  miscellaneous  data 

These  factors  may  be  considered  the  independent  variables.  Indeed,  the  factors  may  not  be  truly 
independent;  their  values  may  be  altogether  constrained  by  some  other  limiting  function  (i.e.,  total 
resources,  or  an  annual  budget).  Nevertheless  in  the  exploratory  stage,  the  analyst  usually  accepts 
the  independency  assumption  and  then  challenges  it  with  the  benefit  of  more  information  in  the  opti¬ 
mization  stage.  In  the  optimization  stage,  appropriate  constraints  can  be  employed  for  searching  to 
define  explicitly  the  experimental  region. 

In  the  initial  exploratory  stage  with  a  model  like  SOAR,  the  number  of  computer  runs  can  cost 
a  prohibitive  amount.  An  experimental  design  is  required  that  will  provide  information  on  all  the 
variables  of  main  interest,  but  can  be  executed  to  unfold  successively  more  information  about  the  more 
significant  variables.  The  factorial  experiment  can  be  used  to  do  this. 

The  first  step  in  the  design  of  the  SOAR  experiments  was  to  eliminate  from  consideration  vari¬ 
ables  of  indirect  interest,  or  variables  that  could  be  analyzed  by  use  of  other  means.  “System  alert 
time”  and  "miscellaneous  data”  were  thereby  eliminated  leaving  21  variables  to  be  considered.  The 
second  step  was  to  decide  on  the  levels  to  be  assigned  to  the  variables.  For  exploratory  purposes, 
each  variable  was  assigned  a  base  or  normative  level  and  an  augmented  level  except  for  the  aircrew 
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variable  which  was  assigned  a  base  level  and  a  decremented  level.  Each  combination  of  variable 
levels  produced  a  design  point.  A  full  factorial  experiment  requires  computer  runs  for  each  design 
point.  Enough  information  is  provided  to  determine  the  effects  of  changing  the  level  of  each  variable 
and  the  effects  of  variable  interactions.  However,  the  information  advantage  in  the  SOAR  analysis 
of  the  full  factorial  was  clearly  offset,  and  infeasible,  by  the  cost  of  making  the  computer  runs  to  -sriisfy 
the  full  factorial  design.  The  number  of  runs  included  in  a  full  factorial  experiment  with  21  variables 
etch  at  two  levels  is  22\  or  2,097,152. 

Two  alternatives  to  the  full  factorial  experiment  were  examined:  fractional  factorial,  and  factor 
aggregation.  The  fractional  factorial  experiment  is  to  make  computer  runs  on  a  subset  of  the  runs  re¬ 
quired  for  a  full  factorial.  The  analyst  selects  a  sequence  of  runs  with  the  factors  at  various  base  and 
augmented  levels.  Although  the  problem  of  factor  choice  during  an  experiment  is  well  documented  in 
the  literature,*  it  poses  a  difficult  problem  for  the  analyst  at  the  cutset.  Thus,  it  was  decided  to  employ 
a  factor  aggregation  method. 

The  factor  aggregation  method  is  to  group  similar  variables  which  are  then  ail  set  at  base  levels  or 
augmented  levels  during  any  one  run.  A  full  factorial  experiment  is  run  using  these  variables  groupings 
as  if  they  were  variables  themselves.  In  the  SOAR  analysis,  variables  were  initially  grouped  into  three 
categories  and  full  factorial  experiments  were  satisfied.  Figure  1  illustrates  the  general  design  set.  The 
zero  (0)  entry  indicates  the  base  level  and  a  (1)  entry  indicates  the  changed  level. 

In  Figure  2,  the  same  design  is  shown  with  the  actual  variables  and  levels  of  a  specific  case  sub¬ 
stituted  for  the  general  notation.  Weather  is  Variable  One,  with  winter  weather  representing  the  (0) 
entry  and  summer  weather  the  (1)  entry.  Variable  Two  is  now  All  Resources,  with  (0)  being  the  “base 
case”  and  (1)  the  changed  (augmented)  case.  Variable  Three  is  Air  Traffic.  In  the  base  case  (0  level), 
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Flc.tjRE  1.  General  Design  Set 


Encounter 

Weather 

All  Resources 

Traffic 

1 

Winter 

Base  level 

MAC  +  CRAF 

2 

Summer 

Base  level 

MAC+CRAF 

3 

Winter 

Augment  50% 

MAC  +  CRAF 

4 

Summer 

Augment  50% 

MAC+CRAF 

5 

Winter 

Base  level 

MAC + CRAP + TAG 

6 

Summer 

Bnse  level 

MAC. + CRAF+ TAC 

7 

Winter 

Augment  50% 

MAC  +  CRAF+TAC 

8 

Summer 

Augment  50% 

MAC  +  CRAF+TAC 

Ftcur.E  2.  Typical  Experimental  Design  Set 


’John  L.  Overholt,  “Factor  Selection,”  in  Thomas  H.  Naylor,  editor,  op.  cit.,  pp.  59-79. 
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only  CRAF  airframes  compete  with  mission  aircraft  for  resources;  in  the  augmented  case  (1  level), 
TAC  (Tactical  Air  Command)  activity  is  added  to  the  MAC  (Military  Air  Command)  and  CRAF  traffic. 

The  impact  of  the  categories,  or  independent  variables,  on  the  dependent  variable,  either  aircraft 
utilization  rate  or  productivity,  was  assessed  by  applying  a  standard  analysis  of  variance  technique. 
Three  factor  analysis  of  variance  with  interactions  shows  the  statistical  significance  of  the  change  in 
resource  level  and  the  effect  of  factor  interaction. 

The  statistical  analysis  of  the  23  factorial  experiment*  is  used  to  determine  whether  further 
experimental  analysis  of  a  category  is  warranted.  If  a  category  is  statistically  significant,  a  second  ex¬ 
perimental  design  is  then  formulated  to  test  that  category  again  with  other  categories.  Alternatively, 
the  category  may  be  parsed  into  two  or  more  less  aggregated  variables  in  order  to  discover  the  actual 
variables  that  are  contributing  to  the  dependent  variable.  On  the  other  hand,  statistically  insignificant 
categories  may  be  temporarily  dropped  from  further  analysis. 

For  instance,  as  it  became  obvious  that  the  variable  All  Resources  had  a  significant  impact,  it  was 
necessary  to  determine  which  particular  resource  types  were  the  most  important  marginal  contributors. 
Therefore  Refueling  and  Maintenance  were  grouped  in  a  separate  category  as  were  Aircraft  Acceptance 
Rates  and  Parking  Space.  These  two  categories  were  then  run  in  a  subsequent  design  set  with  Weather 
providing  the  third  variable.  If  the  experiments  demonstrated  that  a  particular  variable  had  no  signifi¬ 
cant  effect,  it  was  not  included  in  future  sets.  The  same  procedure  was  used  when  it  was  determined 
that  enough  information  had  been  gathered  about  a  particular  variable.  For  example,  in  the  SEAsia 
case  Weather  was  a  variable  in  16  runs.  In  the  last  six  runs  (a  total  of  22  SEAsia  encounters  were  run  in 
the  experiments)  only  winter  weather  was  used  since  it  was  deemed  that  enough  information  concerning 
the  marginal  contribution  of  seasonal  weather  changes  had  been  collected. 

SOUTHEAST  ASIA  ANALYSIS 

Twenty-two  computer  simulations  were  made  with  the  SOAR  model  for  the  SEAsia  case.  In  these 
runs,  the  objective  was  to  quantify  the  impact  on  the  airlift  system  of  such  variables  as  weather,  base 
resources,  competing  traffic,  aircrews,  and  mission  aircraft.  The  impact  of  changes  in  the  levels  of  the 
variables  was  measured  in  terms  of  their  effect  on  utilization  rates  and  productivity.  Three  different 
tools  of  statistical  analysis  were  employed:  three  factor  analysis  of  variancet  was  used  primarily  to 
test  the  level  of  significance  of  the  changes  in  the  dependent  variables  caused  by  the  factors  and 
interactions  being  evaluated;  paired  comparisons  were  used  to  identify  the  marginal  contributions 
of  the  variables;  and  regression  analysis  was  used  to  examine  the  relationship  of  utilization  rate  to 
productivity. 

Figure  3  illustrates  the  independent  variables  which  were  changed  during  the  22  SEAsia  runs. 

In  the  first  experiment,  the  three  factors  were  Weather,  Cargo-handling  Resources  and  Other 
Base  Resources;  for  Weather,  the  base  level  was  winter;  the  changed  level  was  summer.  For  the 

*2J<=*8,  the  number  of  possible  combinations  of  two  levels  for  each  of  the  three  factors  being  tested.  Each  SOAR  run  ta 
equivalent  to  one  observation  in  the  2s  factorial  experiment. 

tThe  use  of  three  factor  analysis  of  variance  is  an  unconventional  use  of  a  familiar  statistical  technique.  Among  staristi- 
clans,  there  is  a  difference  of  opinion  as  to  whether  this  technique  is  appropriate  to  the  purpose  to  which  it  was  applied.  The 
adequacy  of  eight  observations  (SOAR  runs)  to  test  the  three  variables  and  their  interactions  is  the  point  in  question.  The  eight 
observations  provide  a  totei  of  seven  degrees  of  freedom.  Of  these,  one  each  is  used  for  each  variable  and  interaction.  The  re¬ 
maining  one  degree  of  freedom  is  applied  to  experimental  error  which  also  includes  the  high  order  interaction.  Many  statisticians 
would  insist  on  more  than  one  degree  of  freedom  for  experimental  error.  The  point  in  question  is  the  power  of  the  test,  the  level 
of  confidence  one  may  place  in  the  results  of  the  test. 
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Figure  3.  SEAsia  Experiments 


two  other  categories,  the  base  level  consisted  of  those  men,  equipment,  materiel,  facilities,  or  capability 
assigned  to  each  airbase.  In  the  changed  case,  the  base  level  of  resources  was  augmented  at  eveiry 
airbase  by  a  constant  percentage,  usually  50  percent.  The  results  of  the  first  experiment  we  shown 
in  Figure  4  for  the  G— 141  and  C~5A. 

The  presence  of  the  word  "none”  in  Figure  4  indicates  that  the  factor  or  interaction  was  not 
statistically  significant  and  that  any  impact  thought  to  be  associated  with  it  most  likely  occurred  by 
chance.  In  fact,  the  level  of  significance  indicates  the  probability  that  the  effect  being  measured  oc¬ 
curred  by  chance.  Thus,  the  smaller  the  number,  the  more  confident  we  can  fee!  that  the  factor  being 
tested  actually  was  associated  with  the  impact  that  was  reflected  in  utilization  rates  or  productivity. 
For  example,  looking  at  C-5A  utilization  rate,  the  0.08  level  of  significance  indicates  a  probability  of 
8  percent  that  the  impact  occurred  through  random  effects  and  a  92  percent  probability  that  at  may 
be  attributed  to  weather.  Furthermore,  by  consulting  the  Marginal  Contribution  chart,  one  sees  that 
the  mean  difference  between  summer  ar.d  winter  weather  was  0.75  hours  (45  mir,  par  aircraft  day). 
Other  Base  Resources  also  affected  utilization  rates;  the  effect  was  statistically  significant  at  the  0.05 
level.  Only  Cargo-handling  seemed  to  have  no  significant  impact  or.  C-SA  rates.  The  C-5A  rates 
were  also  relatively  sensitive  to  the  interaction  of  the  variables,  although  there  was  a  good  chance 
that  such  an  impact  could  have  occurred  through  random  effects  {25  to  3?  percent). 

The  C— 141,  on  the  other  hand,  was  not  sensitive  to  any  changes  in  the  factors  or  interactions. 
With  the  single  exception  of  the  effect  of  weather  on  produc'-vny,  there  is  reason  to  believe  the  dif¬ 
ferences  which  were  registered  actually  occurred  by  chance. 

In  Experiment  3 ,  Cargo-handling  was  shown  to  have  a  negligible  impact  on  both  aircref5.  Weather 
and  Other  Base  Resources,  however,  had  an  important  effect  on  C-5A  utilization  rates  and  e  strong 
indication  for  C-5A  productivity.  Therefore  Weather  was  retested  in  Experiment  2.  Other  Base  Re- 
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Marginal  contributions 
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None 
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0.75 

BC.  Interaction . 

0.75 

L_  ... 

0.75 

0.25 

None 

Figure  A,  Experiment  1 


sources  was  subdivided  into  two  more  definitive  categories:  (1)  Refueling  and  Maintenance,  and 
(2)  Airfield  Acceptance  Capability  and  Parking.  The  results  of  Experiment  2  arc  shown  in  Figure  5. 

With  the  possible  exception  of  Airfield  Acceptance  Capability  and  Parking,  none  of  the  categories 
in  Experiment  2  were  shown  to  be  significant  at  a  meaningful  level.  This  evidence  appeared  to  confirm 
the  indications  of  Experiment  1  that  the  levels  of  resources  in  the  base  case  were  too  high  for  the  level 
of  demand.  As  a  result  it  was  decided  to  stress  the  system  by  increasing  demand. 

In  Experiment  3,  Weather  was  tested  once  again.  All  Base  Resources  was  the  second  variable 
tested  (it  actually  was  the  second  test  for  this  category  since  in  Experiment  1 ,  Cargo-handling  Resources 
and  Other  Base  Resources  add  up  to  All  Base  Resources).  The  third  category  was  Competing  Aircraft. 
In  the  base  case,  competing  aircraft  consisted  of  commercial  carriers  (CRAF)  only.  In  the  augmented 
case,  the  deploying  tactical  air  forces  (TAG)  were  added  to  CRAF.  Results  appear  in  Figure  6. 

Once  more,  the  experiment  illustrated  the  relative  insensitivity  of  C— 141  to  changes  in  the  operat¬ 
ing  environment.  The  increased  competition  for  resources  prol-sbiy  had  such  a  negligible  effect  that  it 
was  entirely  overcome  by  random  effects  which  produced  an  increase  in  utilization  r"tes  and  pro¬ 
ductivity  for  the  C-141.  The  highest  utilization  rate  for  all  22  SOAR  runs  was  recorded  in  this  set; 
tie  C-14Is  averaged  15.19  hours  in  the  Summer  case  having  aevmented  resources  and  the  fully  aug 
uiented  level  of  competing  aircraft. 

The  greatest  impact  for  the  categories  being  tested  were  fouod  in  C-5A  productivity.  All  of  the 
factors  and  interactions  were  determined  to  have  a  statistically  «ign.(kunt  impact  at  levels  ranging  from 
0.005  to  0.10.  The  addition  of  competing  aircraft  resulted  in  the  loss  of  an  average  of  1.17  hours  and 
6,690  ton  miles  per  aircraft  per  day. 
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Marginal  contributions 
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Figure  5.  Experiment  2 


Marginal  contributions 
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Figure  6,  Experiment  3 
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In  Experiment  4,  Weather  was  dropped  from  further  consideration  because  sufficient  data  had  been 
accumulated:  it  accounted  for  an  average  marginal  increase  in  productivity  for  “C-5As”  of  6,333  ton 
miles  per  day  and  1,77 1  ton  miles  per  day  for  C-141s.  In  order  to  observe  the  system  under  maximum 
stress.  Mission  Aircraft  was  evaluated  as  a  variable.  In  the  base  case,  the  number  of  C-5As  and 
C-141s  were  67  and  157,  respectively;  in  the  augmented  case,  the  C-5As  were  increased  to  96  and  the 
C-141s  to  224.  The  second  category  tested  was  All  Base  Resources.  Flight  Crews  constituted  the  third 
category.  In  the  base  case,  1,464  crews  were  used;  in  the  changed  case  (decremented,  this  time)  the 
number  was  1,246.  The  results  of  Experiment  4  are  shown  in  Figure  7. 
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Figure  7,  Experiment  4 


As  expected,  the  increased  demand  generated  by  the  additional  mission  aircraft  caused  the 
category  “All  Base  Resources”  to  register  a  more  impressive  impact.  It  was  statistically  significant  at 
levels  ranging  from  0.10  to  0.20.  Changes  in  the  levels  of  “All  Base  Resources”  accounted  for  a  variation 
of  2,20  hours  and  23,640  ton  miles  per  day  for  each  C-5A.  Even  the  relatively  insensitive  C-141 
varied  an  average  of  0.87  hours  and  3,700  ton  miles  per  day  in  response  to  changes  in  “All 
Base  Resources.” 

The  addition  of  more  mission  aircraft  caused  ■?  decline  in  utilization  rates  and  productivity  per  air- 
craf  This  is  a  further  illustration  of  the  effect  of  queueing  brought  on  by  increased  competition.  It  is 
also  an  indication  that  airlift  forces  productivity  is  not  directly  proportional  to  the  number  of  aircraft 
applied.  That  is;  the  productivity  of  one  C-141  may  be  60,000  ton  miles  per  day,  but  the  productivity  of 
100  C— 14 Is  is  probably  less  than  6  million  ton  miles  (i.e.,  100x6,000). 
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EUROPE/ AFRICA  ANALYSIS 

Twenty  computer  simulations  were  made  with  the  SOAR  model  for  the  simultaneous  deployments 
to  Europe  and  Africa.  Because  these  runs  were  made  after  the  SEAsia  runs  had  been  completed,  they 
were  designed  with  the  benefit  of  more  experience  with  the  computer  model  and  a  better  understanding 
of  system  behavior.  Thus,  it  was  possible  initially  to  focus  these  experiments  on  specific  resources. 
In  addition,  after  the  first  design  set,  augmentation  was  limited  to  those  resources  or  capabilities  which 
could  reasouJjly  be  expected  to  be  dramatically  improved.  In  the  dual  deployment,  15  C-5A  and  105 
C-141  aircraft  were  allocated  to  the  African  deployment;  the  remaining  52  C-5As  and  42  C-141s 
were  assigned  to  European  requirements.  Figure  8,  a  summary  chart  of  the  20  SOAR  encounters,  shows 
the  variables  tested  in  the  Europe/Africa  runs. 
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Figure  8.  Europe/ Africa  Cate 


As  before,  eight  SOAR  encounters  were  used  to  provide  the  observttions  for  the  2s  factorial  design 
sets.  Three  experiments  were  executed,  In  Experiment  1,  the  variables  tested  were:  Weather,  Flight 
Crews,  and  All  Airbase  Resources.  Once  again,  the  base  Weather  level  consisted  of  winter  wind  factors 
and  airfield  observations;  summer  provided  the  changed  level.  For  Flight  Crews,  the  p.-ogrammed 
4.42  crew  ratio  was  used  for  the  base  level;  the  changed  (augmented  level)  was  a  5.42  crew  ratio.  For 
the  base  level  for  All  Airbase  Resources,  the  study  group’s  best  estimate  of  men,  equipment,  materiel, 
and  capability  available  at  the  system’s  airbases  was  used.  For  the  augmented  level,  these  resources 
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were  increased  system-wide  by  a  flat  percentage,  usuaiiy  approaching  SO.  The  results  of  Experiment  1 
are  shown  in  Figure  9. 


Marginal  contributions 


Category 

C-IO 

C-5A 

Utilization 
rate  (hours 
per  day) 

Ton-mile 
productivity 
per  day  (000) 

Utilization 
rate  (hours 
per  day) 

Ton-mile 
productivity 
per  day  (000) 

A.  Crew* . . . 

| 

-0.25 

-0.30 

-0.31 

B.  Weather . 

0.59 

-0.15 

-1.1 

.  C.  All  Airbase  Resource* . 

mmSmm 

13.49 

3.59 

48.69 

Levels  of  significance 


Category 

C-141 

C-5A 

Ute  rate 

Productivity 

Ute  rate 

Productivity 

A.  Crews . . . . . 

None 

None 

0.10 

None 

B.  Weather . . . . . 

None 

0.75 

0.20 

None 

C.  All  Airbase  Resources . 

0.05 

0.05 

0.005 

0.05 

AB.  Interaction . . . 

None 

None 

o.os 

0.50 

AC.  Interaction . . . 

0.75 

None 

0.10 

None 

BC.  interaction . . . 

0.75 

None 

0.05 

None 

FtcuJU'  9.  Experiment  1 


The  results  of  the  paired  data  tests  used  to  compute  marginal  analysis  indicate  that  seasonal 
changes  in  weather  had  little  effect  on  utilization  rates.  Since  one  might  anticipate  a  significant  effect 
from  weather  in  northern  Europe,  the  suspicion  arises  that  the  weather  enroute  to  Africa  somehow 
dampened  or  washed  out  the  effects  of  European  weather.  This  hypothesis  was  supported  by  a  separate 
analysis  of  weather  effects  by  geographical  area. 

The  negative  effects  of  the  addition  of  one  extra  flight  crew  per  aircraft  may  be  attributed  to  random 
effects.  Apparently  the  base  level  crew  ratio  w«o  sufficient  to  support  airlift  activity  at  the  levels 
tested;  the  additional  crews  were  neither  required  nor  used.  Therefore,  the  net  effect  was  that  of 
additional  runs  of  the  base  cases  with  random  effects  providing  a  negligible  variation  in  results.  Augmen¬ 
tation  of  All  Airbase  Resources  resulted  in  a  dramatic  increase  in  both  utilization  rates  and  productivity. 
The  impact  was  significant  at  levels  ranging  from  0.05  to  0.005.  The  important  increases  resulting  from 
the  augmentation  of  All  Base  Resources  indicated  that  a  more  detailed  examination  of  this  aggregated 
category  might  be  productive.  Accordingly  Experiment  2  was  designed  to  test  at  two  levels  the  follow¬ 
ing  variables:  Base  Parking  Space,  Refueling  and  Maintenance  (combined),  and  Weather.  All  other 
variables  were  set  at  base  levels.  Results  of  Experiment  2  appear  in  Figure  10. 

The  evidence  shown  in  Marginal  Contributions  for  Experiment  2  reinforces  the  indications  from 
Experiment  1.  The  important  contribution  made  by  All  Airbase  Resources  in  Experiment  1  is  essen¬ 
tially  duplicated  by  the  combined  contribution  of  Refueling/Maintenonce  and  Parking.  The  aggregate 
category,  Refueling/Moinienance,  is  shown  to  be  an  important  marginal  contributor;  the  average  differ¬ 
ence  between  the  base  level  productivities  and  the  augmented  level  of  productivities  is  shown  to  be 
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Marginal  contribution* 


C— 141 

C-5A 

Utilization 

Ton-milc 

Utilization 

Ton-mile 

rate  (hour* 

productivity 

rate  (hours 

productivity 

Category 

per  day) 

per  day  (000) 

per  day) 

per  day  (000) 

A.  Refuel/Maintenince . 

2.88 

11.59 

2.73 

33.88 

B.  Weather . 

0.05 

0.49 

0.17 

1.61 

C.  Parking . 

0.17 

1.44 

0.62 

7.87 

Level*  of  significance 


Category 

- — 

C— 141 

C-5A 

Ute  rate 

Productivity 

Ute  rate 

Productivity 

A.  Refuel/Maintenance . 

0.08 

0.06 

0.08 

0.10 

B.  Weather . 

0.50 

0.75 

0.75 

0.75 

C.  Parking . 

0.75 

0.25 

0.25 

0.35 

AB.  Interaction . 

0.60 

None 

0.50 

0.60 

AC.  Interaction . 

0.50 

ass 

0,75 

0.75 

BC.  Interaction . 

None 

0.75 

0.75 

0.75 

Figure  10.  Experiment  2 


Marginal  contributions 


Category 

— 

C— 141 

C-5A 

Utilization 
rate  (hours 
per  day) 

Ton-mile 
productivity 
per  day  (000) 

Utilization 
rate  (hours 
per  day) 

Ton-mile 
productivity 
per  day  (000) 

A.  Refueling . 

2.59 

10.66 

2.42 

29.64 

B.  Cargo-handling . 

0.06 

0.61 

0.06 

0.00 

C.  Maintenance . 

-0.03 

-0.21 

-0.23 

-8.80 

Level*  of  significance 


Category 

C— 141 

C-5A 

Ute  rate 

Productivity 

Ute  rate 

Productivity 

A.  Refueling..... . . . . . 

0.025 

0.01 

0.05 

0.08 

B.  Cargo-handling . . . 

0,40 

u.20 

0.75 

None 

C.  Maintenance . 

None 

0.50 

0.40 

0.50 

AB.  Interaction . 

0.40 

0.20 

0.90 

None 

AC.  Interaction . . . 

Q.2S 

0.10 

0.75 

0.75 

BC.  Interac.iuu . 

0.75 

0,35 

A 

V.tU 

0,75 

Figure  11.  Expenses!  3 
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11,590  ton  miles  per  aircraft  day  and  33,880  ton  miles  for  the  C-141  and  C-5A,  respectively.  The  impact 
of  this  change  is  statistically  significant  at  levels  ranging  from  0.10  to  0.05.  The  marginal  contribution 
of  additional  parking  space  is  slight,  indicating  that  the  amount  of  parking  presently  available  would 
not  be  a  serious  constraint  on  utilization  rates  ranging  from  8.88  hours  to  13.46  hours. 

Once  again  Weather  was  found  to  be  a  negligible  factor.  Therefore,  Weather  was  not  tested  in 
the  third  and  last  experiment.  Because  of  its  slight  impact,  Parking  was  also  dropped  from  further 
consideration.  But  Refueling  and  Maintenance  were  separated  with  each  providing  one  variable  in 
Experiment  3.  The  third  variable  was  Cargo-handling  The  results  of  the  third  Experiment  appear  in 
Figure  11. 

In  Experiment  3,  all  experiments  were  run  with  winter  weather.  With  the  exception  of  those  varia¬ 
bles  being  tested,  all  resources  and  capabilities  were  held  tothebaselevels.  The  results  indicate  that 
Refueling  capability  is  the  key  variable  in  the  Europe/Africa  depolyment.  Experiment  1  identified  the 
important  and  significant  contribution  played  by  All  Airbase  Resources.  In  Experiment  2,  the  identity 
of  the  critical  resource  was  narrowed  down  to  the  aggregate  category  Refueling  and  Maintenance. 
In  Experiment  3,  it  was  revealed  that  Refueling  was  the  single  most  constraining  factor  in  the  system. 
The  next  logical  step  in  the  analytical  process  would  be  to  identify  those  key  bases  in  the  network  at 
which  refueling  is  a  serious  problem.  It  could  well  be  that  the  addition  of  a  dozen  or  so  refueling  trucks, 
relatively  easy  to  deploy,  could  eliminate  the  choke  point  and  make  possible  significant  increases  in 
airlift  effectiveness.  Also,  a  change  in  the  number  or  scheme  for  recovery  bases  may  alleviate  the 
problem. 

CONCLUSION 

The  simulation  findings  were  instrumental  in  arriving  at  the  joint  OSD/Air  Force  determination 
that  while  some  increase  in  aircraft  utilization  rates  could  be  achieved  by  improvements  in  system 
support  resources,  it  would  not  be  feasible  to  operate  the  airlift  force  at  the  proposed  utilization  rate 
during  a  wartime  deployment.  The  simulation  was  especially  valuable  in  identifying  constraints  in  the 
system  and  in  estimating  the  kinds  and  levels  of  resource  augmentation  required  to  relieve  the 
constraints. 

In  addition  to  the  analysis  of  the  effect  of  individual  resources,  an  examination  of  several  relation¬ 
ships  commonly  assumed  to  exist  between  measures  of  airlift  effectiveness  and  certain  system  variables 
was  conducted.  Two  of  the  most  interesting  findings  concerned  the  assumptions  of  direct  proportion¬ 
ality  between  productivity  and  utilization  rate  and  between  productivity  and  numbers  of  aircraft 

Productivity  is  generally  recognized  as  the  key  measure  of  effectiveness  because  of  its  obvious, 
direct  impact  on  mission  accomplishment.  The  relationship  between  productivity  and  utilization 
rates  (an  interim,  easily  quantifiable  measure),  has  long  been  assumed  to  be  one  of  direct  proportion¬ 
ality.  Regression  analysis  was  employed  to  determine  the  relationship  between  the  two  variables. 
The  relationship  was  shown  to  be  linear  wi''^n  the  range  of  the  values  tested.  The  assumption  of 
direct  proportionality,  however,  was  shown,  by  the  presence  of  a  constant,  to  be  an  incorrect  one. 
The  size  and  sign  of  the  constant  were,  in  turn,  found  to  be  a  function  of  aircraft  type  and  deploy¬ 
ment  area.  The  regression  equations  suggest  the  existence  of  an  appropriate  predictive  expression 
for  eaeh  aircraft  type  and  specific  contingency  ar.a.  The  development  and  use  by  contingency  plenners 
of  a  family  of  predictive  equations  would  result  in  more  accurate  estimates  of  productivity,  closure 
dates,  and  airframe  requirements. 
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Direct  proportionality  haa  also  been  assumed  between  the  productivity  of  an  airlift  force  and  the 
nr  .iibers  of  aircraft  in  that  force.  Thus,  if  the  daily  productivity  of  one  C-141  is  54  kiloton  miles, 
then  the  productivity  of  10  is  540  kiloton  miles  and  the  productivity  of  100  is  5,400  kiloton  miles.  The 
assumptions  underlying  such  calculations  are  that  marginal  increases  per  aircraft  are  uniform  regard¬ 
less  of  the  size  of  the  airlift  force.  As  expected,  this  assumption  has  been  found  to  be  incorrect  As 
aircraft  are  added  to  the  base  programmed  level,  total  productivity-rises,  . but  per  aircraftTprodtfclivity 
declines.  The  important  point  here  is  the  provision  of  a  method  to  quantify  an  alternative  assumption 
to  the  direct  proportionality  assumption. 

With  the  capability  to  cope  with  extremely  complex  aspects  of  dynamic  systems  that  are  clearly 
beyond  the  capability  of  most  mathematical  models,  simulation  has  been  demonstrated  to  be  a  valuable 
technique  for  airlift  system  analysis.  The  key  to  centre-ling  a  large  simulation  analysis,  such  as  the 
SOAR  analysis,  resides  in  a  carefully  constructed  experimental  design  which  provides  information 
in  a  structured  manner  and  which,  at  the  same  time,  permits  the  flexibility  for  the  analyst  and  decision¬ 
maker  to  test  new  hypotheses  developed  from  additional  information  that  comes  available  during.the 
analysis.  A  useful  technique  developed  for  thi  analysis  js  to  design  experiment  sets  for  a  finite  number 
of  factors.  Initial  analysis  begins  with  aggregated  sets  of  factors,  and  information  ga;ned  from  experi¬ 
ment  execution  is  used  to  parse  sets  and  ultimately  identify  major  explanatory  independent  variables. 
Moving  from  aggregate  to  the  individual  variables  also  permits  overlap  of  computer  runs  for  experi¬ 
ments.  Thus,  the  technique  facilitates  obtaining  the  maximum  information  from  the  minimum  number 
of  computer  runs. 
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